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1.  Introduction  Let  the  process  X(t),  be  defined  by 

dX(t)  -  u(X(t))dt  +  d!tf(t) 

where  W(t)  is  a  standard  Wiener  process  and  u  is  a  control  function, 
and  X(0)  »  a. 

Let  9  be  even,  convex,  symmetric  positive,  exponentially  bounded 
and  strictly  increasing  on  the  positive  axis.  Let  the  average  expected 
cost  function  be 

T 

-  Jzrm  ;J  E  [?(X(s»  +  |u  (X(s))ltfJ  ds 
0 

with  *  >  0  . 

The  object  is  to  find  the  control  law  u  which  minimizes  J 
subject  to  a  <  0  <  b  where  a  <  0  <  b  are  real  numbers.  A  two-dimensional 
version  is  indicated.  The  case  or  •  1  is  considered  in  great  detail  in 
[1].  The  results  for  the  cases  «  4  l  are  given  in  this  paper.  The 
ccsqplete  proofs  are  as  in  [1]  and  reference  is  made  to  that  paper  for 
the  complete  proofs  of  the  results  mendoned  here.  In  fact,  it  will 


2. 


suffice  to  indicate  the  nature  of  the  solutions  to  the  "asymptotic 
dynamic  programming  equation"  in  one-or  multi-dimensions  in  [1],  since 
the  other  arguments  are  as  given  there,  or  sufficiently  similar  so  as 
to  not  be  explicitly  given.  In  one  dimension  it  has  the  form 


then 


CS5i)C  ¥0  if  c<Oa„d(M)  < 


(.  or  or  >. 

0,  ac  +  |a|  ,  be  +  Jb|  j  otherwise 


Proof  The  minimum  is  either  at  an  interior  differentiable  point  or 
at  one  of  the  three  points  0,  a,  b. 

Theorem  1  Assume  a  <  0  <  b  . 

(7)  If  d  -  2n,  a  >  1 

Then  there  are  distinct  numbers  x^  <  xjj  such  that  for 
<  x  <  x2 ,  the  "asymptotic  dynamic  programming  equation  [  1]  ,  is 


For  x  > 


f"  2n 

X  -  |  +  af  •  +  aza  +  9  . 


The  x^,  Xj,  X  are  found  by  continuity  at  the  boundaries  and  by 
setting  f'Cxj)  -  f'(»)  or  f'(Xj)  ■  f’(-*)  . 

(b)  If  a  i*  2n,  am*  1 


4 


Then  there  are  distinct  numbers  y^,  1  <  l  <  3 

y^  <  0  <  <  y^»  such  that  the  "asymptotic  dynamic 

programming  equations  [l]"  satisfy,  if 

x  <  yL  , 

f«  * 

X  -  |  +  bf'  +  jb  1  +9 


and  if 
then 


and  if 
then 


yL  <  x  <  0, 


X  -  |M  +  af*  +  |a|  +q> 


0  <  *  <  y„ 


_1_ 

CK-l 


-r+op") 


and  if 


72  <  x 


then 

f"  i  ,a 

X»2  +  af '  +  |a|  +?, 

where  the  values  y^,  yj»  y^,  are  obtained  by  setting  the 
solutions  equal  (by  continuity)  at  y^ ,  ,  y^  and  f(y^)  *  f(®) 


(6) 

or 


£(yL>  -  £(-->• 

Proof  The  axjpressions  (5),  (6)  follow  from  (1),  (2)  using 

a 

monotonicity  and  symmetry  of  9,  the  properties  of  c  u  +  }u| 
as  a  function  of  c,  u,  the  consequent  symmetry  of  f't  and  the 
uniqueness  arguments  in  [1]. 


Theorem  2 

0  M  =, 


2n,  if  a  <  0  <  b  , 


-€£k%*£ 


if 


the  optimal  law  is 


u(X)  - 


© 


1 

2n-l 


CW 


,  x  1  <  X  <  x2 


»  X  <  x2 

,  X  >Xz 


If  0  <  a  #  1,  a  4  2n,  then  for  a  <  0  <  b. 


u(X)  - 


cw 


if  0  <  X  <  y_ 


if  X  <  yx 

if  y2  <  X  <  0  or  y2  <  X  (8) 


Proof  This  follows  immediately  from  Theorem  2  and  the  lernmas ,  (3)  -  ( 
Details  are  as  in  [1]  for  uniqueness  and  optimality  of  u. 


3.  Multi-dimensional  case -Remarks 


Let  Xft)  be  a  k  x  1  stochastic  process  with 
d  $i*l  -  tt(x(t))dt  +  dw(t) 

where  u  is  d  x  1  and  W  is  d  x  1  Wiener  process  with  independent 
components  and  X  (0)  ■  x  . 


1  I 

J(x,u)  -  t~wT  J  e[9  (X(s»  +  llu(X(s))Hp]ds 

where  «p  la  even,  convex,  positive  and  exponentially  bounded  in 

each  argument,  and 


6. 


1 

P 


F  X 

£  lu.j  )  for  some  p  >  0  . 

A_1  *  * 


d  i  p 

-  (  £  |u  . 

p 

It  is  desired  to  minimize  J  subject  to 


*A<  “x<bi 


1  <  l  <  d 


where  a^,  b^. 


I  <  l  <  d  are  real  numbers.  This 
constraint  will  be  written  a  <  u  <  b  . 


Let  c  ■  (c1,...cd>)  and 
denote 

h(c) 


min  (c*u  +  Uulj  )  . 
a  <  u  <  b  p 


(11) 


The  evaluation  of  h(c)  requires  the  evaluation  of  c*u  + 
at  the  2+1  points  given  by  u^  *  a^  or  b^,  1  <  £  <  d  and  at  the 
u  minimizing  c-u  +  j|u|]^,  for  fixed  c.  As  c  varies,  Rd  is  cut 
into  regions  defined  by  the  minimum  h(c)  and  the  u  achieving  that 
minimum. 

Let  f  -  (f<l>(x),...,f(d)(^)  for  x-  (xx . xd) 


such  that 


*  yggy  exists  and  is  continuous  1  <  i,  j  <  d  . 


"i  "J 


Let  «  (x)  -  # 


J  ~  ^ 


j 


to r  the  various  regions  above,  it  is  required  to  solve  (for 
minimum  X  ) 

X  -  j  £  fij(2)  +  h(fi(S)»**»£«j(2£)>  +  *  (*)  • 


(12) 


Equating  solutions  at  the  boundary  of  these  regions  defines  a 


finite  number  of  vectors  x,  =  (x. ,  .),  x_  ,...,x  where 

l  It  Id  —  £  —  n 

,d 


n  =»  2  +1  such  that  f.(x  )  "  f.(°»  )  where  =*  denotes  any  ray  in 

j  —  r  j  r  r  1  J 

a  boundary  region  such  that  at  least  one  coordinate  may  be  set  equal 
to  +  00  and  the  point  so  obtained  remain  in  the  region.  This  determines 
X  also. 


It  follows  as  before  that  if  a  optimal  u  ,  then 


u0<p  »  that  u  giving  h(f  ,...,fd> 


in  any  of  the  regions  given  by  the  vectors  as  edges. 


Remark  2  If  in  addition  Hull  <  M  <  and  if  u  ^  *=  optimal  u 
with  this  additional  constraint,  then  with 


if  ll“0lip  <  M 


if  IIhqIIp  *  R  >  w 


(13) 


which  follows  by  a  variational  inequality.  See  [2]  for  a  similar 
argument. 
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